Expression recognition in messaging systems

ABSTRACT

Some embodiments include a messaging system capable of expression-based communication and/or expression-based actions. The messaging system can run on a computing device. For example, the messaging system monitors a camera feed from a camera of the computing device to detect a biometric signature when a messaging interface of the messaging application is actively being used. The messaging system can match the detected biometric signature against a known profile utilizing a facial recognition process to authenticate an operating user to use the messaging application. The messaging system can determine a human expression based on the detected biometric signature utilizing an expression recognition process to associate a contextual tag with an activity on the messaging interface. The messaging system can then communicate with a message server system to associate the contextual tag with content presented to the operating user or a conversation participated by the operating user via the messaging application.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. patent application Ser. No.14/643,810, entitled “EXPRESSION RECOGNITION IN MESSAGING SYSTEMS,”which was filed on Mar. 10, 2015, which claims the benefit of U.S.Provisional Patent Application No. 61/950,423, entitled “BIOMETRIC FORMOBILE ACCESS,” which was filed on Mar. 10, 2014; U.S. ProvisionalPatent Application No. 61/985,059, entitled “USE OF BIOMETRIC FOR ACCESSTO DATA DEVICE AND ASSOCIATED SOLUTIONS FOR DIFFICULT BIOMETRIC READINGSCENARIOS,” which was filed on Apr. 28, 2014; and U.S. ProvisionalPatent Application No. 62/051,031, entitled “EXPRESSION RECOGNITION INMESSAGING SYSTEMS,” which was filed on Sep. 16, 2014; which are allincorporated by reference herein in their entirety.

RELATED FIELD

At least one embodiment of this disclosure relates generally to anelectronic messaging system, and in particular to privacy and securityof an electronic messaging system.

BACKGROUND

With the wide availability of mobile devices, in some areas/cultures,electronic messaging is becoming an integral part of a person's life.Because of this, privacy and security concerns arise over the use ofsuch systems. Conventional technology protects against privacyviolations by providing a screen lock on a mobile device whenever themobile device is not used. An authorized user can unlock the screen bytyping a passcode into the mobile device. However, the passcode isknowledge that is transferable, and hence may be stolen. Furthermore,the screen lock prevents access to other applications on the mobiledevice, making it unnecessarily inconvenient. Other solutions protectagainst violation by scheduling the destruction of a message to ensurethat its content therein does not survive indefinitely.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A is a diagram illustrating a messaging application of a mobiledevice revealing content of a message when a facial profile associatedwith a recipient account is recognized, in accordance with variousembodiments.

FIG. 1B is a diagram illustrating a messaging application of a mobiledevice hiding content of a message when a facial profile associated witha recipient account is not detected, in accordance with variousembodiments.

FIG. 1C is a diagram illustrating a messaging application of a mobiledevice hiding content of a message when a facial profile associated witha recipient account is detected and a second unauthorized facial profileis detected, in accordance with various embodiments.

FIG. 2 is a block diagram of a system environment of a messaging systemimplementing a biometric security mechanism, in accordance with variousembodiments.

FIG. 3 is a flow chart of a method of operating a messaging applicationon a computing device that implements an expression recognition process,in accordance with various embodiments.

FIG. 4 is a flow chart of a method of operating a messaging applicationon a computing device capable of delivering advertisements, inaccordance with various embodiments.

FIG. 5 is a flow chart of a method of operating a message server systemthat facilitates conversations between computing devices, in accordancewith various embodiments.

FIG. 6 is a block diagram of an example of a computing device, which mayrepresent one or more computing devices or servers described herein, inaccordance with various embodiments.

The figures depict various embodiments of this disclosure for purposesof illustration only. One skilled in the art will readily recognize fromthe following discussion that alternative embodiments of the structuresand methods illustrated herein may be employed without departing fromthe principles of the invention described herein.

DETAILED DESCRIPTION

Disclosed is a technology that pertains to protecting messages through amessaging system via a biometric security mechanism. The messagingsystem includes a message server system comprising one or more computerservers and messaging applications running on end-user devices. Theend-user devices can include personal computers, smart phones, tablets,wearable devices, gaming consoles, smart TVs, other electronic gadgets,or any combination thereof. The messaging applications act asindependent agents for the messaging system. For example, a messagingapplication may be installed on a general-purpose operating system(e.g., Windows, android, iOS, etc.). The messaging application can alsobe implemented by client-side script (e.g., JavaScript) that may beexecuted on a web browser of the end-user devices.

The biometric security mechanism is implemented via the messagingapplication. Unlike the conventional security mechanisms for a recipientuser device that protect privacy only of the user of the recipient userdevice, the disclosed biometric security mechanism also protects theprivacy and security of the sender of the message. This enables a newparadigm of protection for stakeholders (e.g., the message senders) whoare conventionally dependent solely on the message recipients to protecttheir interests.

In various embodiments, the disclosed technology uses sensors inend-user devices to monitor and profile an end-user. Such profiling canserve to ensure security (for example, by utilizing biometricrecognition, such as facial recognition or ear recognition). Suchprofiling can also serve to enhance context mining (for example, byutilizing expression recognition). In some embodiments, the expressionrecognition can be implemented without implementing the biometricrecognition. In some embodiments, the biometric recognition can beimplemented without implementing the expression recognition.

Security

The biometric security mechanism implements a biometric recognitionprocess to verify one or more identities as authorized by the messagesenders. For example, the biometric recognition process may be based onfacial recognition, ear recognition, silhouette recognition, speakerrecognition, fingerprint recognition, device motion pattern recognition,contact pattern recognition, etc. The biometric security mechanism candetect biometric patterns utilizing a sensor, such as a camera, amicrophone, an accelerometer, a touch sensor, a gyroscope, etc. Thebiometric security mechanism can then profile (e.g., by recordingpatterns and then training one or more computational functions thatrecognize one or more shared attributes or characteristic of thepatterns) the biometric patterns of one or more users into a profilemodel utilizing machine learning algorithms, such as support vectormachines, Gaussian mixture models, hidden Marcov models, etc. Based onthe profiles of the users, the biometric security mechanism canrecognize a user in real-time by matching the biometric pattern detectedby a sensor with the profile model.

The biometric security mechanism is instantiated whenever the messaginginterface of the message application is actively opened and running onan end-user device (e.g., such that the mechanism is active while themessage application is open). The biometric security mechanism utilizesone or more types of biometric recognition processes to determinewhether the people who have access to the end-user device (e.g., lookingat the screen of the end-user device) are consistent with the privacysettings of the sender account and the recipient account.

To use the messaging application on a user device, a user may have toidentify a messaging system account associated with the messagingsystem. The biometric security mechanism uses the one or more types ofbiometric recognition processes to ensure that an operator using themessaging application matches a biometric profile of the messagingsystem account. This procedure ensures that a third party (e.g., abrother or a classmate) cannot send messages on behalf of the user. Inthe case that there are multiple operators for the same end-user device,this procedure also ensures that independent private sessions of usingthe messaging application can be established based on automaticbiometric recognition.

The biometric security mechanism can provide customizable privacyshields per message or conversation. These privacy shields may becustomized by privacy settings dictated by the sender of the message orthe initiator of the conversation. The biometric security mechanism canuse the one or more types of biometric recognition processes to ensurethat the operator using the messaging application matches an authorizedbiometric profile in accordance with a privacy setting of an individualconversation or message. For example, the default privacy setting may bethat the message or conversation is revealed when a biometric profile ofthe recipient account is recognized by the biometric security mechanism.In another example, the default privacy setting may be that the messageor conversation is revealed only when the biometric profile of therecipient account is recognized without the presence of other detectedhuman beings in the proximate area of the recipient end-user device.

In some embodiments, the privacy setting can indicate a singleauthorized account or biometric profile. In some embodiments, theprivacy setting can indicate a group of authorized accounts or biometricprofiles. In some embodiments, the privacy setting can indicate anattribute of a biometric profile, such as age, gender, facial gesture,facial expression, vocal pitch, etc. The attribute can be the solevariable to authorize a person. For example, the privacy setting canindicate that any female face (e.g., in the case of facial recognition)or any female voice (e.g., in the case of speaker recognition) would beauthorized. The attribute can be an additional conditional on top of aspecific account or biometric profile. For example, a sender account(e.g., John) may send a message with a privacy setting indicating Nancyas the recipient account. In some cases, the privacy setting can add anattribute of the recipient profile being female.

The privacy shield may be implemented, for example, as a screen lock, ascrambling of the message content, a blur of the message content, ablackout of the message content, or any combination thereof. In someembodiments, the biometric security mechanism continuously loops throughthe one or more types of biometric recognition processes. In someembodiments, the biometric security mechanism disengages the privacyshield whenever an authorized biometric profile is recognized anddetected. In some embodiments, the biometric security mechanism engagesthe privacy shield whenever a non-authorized biometric profile isdetected.

A sender account or a recipient account can be associated with a singleindividual or a group of individuals. In the case of a group ofindividuals, biometric profiles of every member are associated with thesender account or the recipient account.

Expression Recognition

The use of the biometric security mechanism provides a dependableauthentication process that improves or guarantees privacy of the usersof the messaging system. Accordingly, the users can trust that thebiometric recognition process is used for protection instead ofexploitation. Because of this trust, an expression recognition processcan be run concurrently to the biometric recognition process. That is, auser, who otherwise would not use a messaging application with anexpression recognition process, would use that same messagingapplication if the biometric recognition process were there to guaranteesecurity and privacy.

The disclosed technology can implement expression recognition process inaddition to the biometric recognition process to provide additionalcontextual information associated with a user's emotional/mood statewhen using a messaging application. The expression recognition of a userof the messaging system can provide several benefits. For example, themessage server system can maintain a stimulus response database thatmaps associations between stimuli presented on the messaging interfaceand expressions recognized by the expression recognition process. Themessage server system can then generate and provide a query interface topresent consumer data (e.g., anonymized consumer data) based on thestimulus response database for advertisers, researchers, or businessintelligence departments.

For another example, the messaging application can use the expressionrecognition process to personalize the messaging application. In somecases, based on the responding expression of a user when viewing aparticular type of message (e.g., from a particular sender or during aparticular time of day), the messaging application can determine whetheror not to present another message of the same type via the messaginginterface.

For another example, the messaging application can use the expressionrecognition process to add gesture control to the messaging application.In some cases, certain facial expressions can correspond to a gesturecontrol to activate or deactivate an interactive component of themessaging application. In one specific example, a frown detected by theexpression recognition process can cause the messaging application tosuspend.

For another example, the messaging application can use the expressionrecognition process to customize targeted advertisement. In some cases,an advertisement may be selected based on the current mood of the user.The current mood of the user, in turn, can be estimated based on theexpression recognized. In some cases, presentation of an advertisementmay be triggered based on a user expression condition. For example, anadvertisement may be shown only when the user is recognized to besmiling.

For another example, the messaging application can use the expressionrecognition process to add context to conversations between users of themessaging system. In some cases, the recognized expression of a senderuser can be added as an emoticon to a message. In some cases, therecognized expression of a viewer user can be fed back to the sender asa status update.

Alternative Embodiments

The disclosure above pertains to the specific example of an electronicmessaging system for delivery of text, images, audio clips, or videoclips. However, it is within the contemplation of this disclosure toimplement the biometric security mechanism to other similar systems thatinclude registration of a sender account and the receiver account. Forexample, the biometric security mechanism can apply to email, socialnetwork, dating network, event/meet up invitation, physical delivery ofgoods, enterprise messaging (e.g., financial agent to customermessaging, real estate agent to customer messaging, medical professionalmessaging, etc.), or any combination thereof. The biometric securitymechanism is advantageous in enforcing privacy settings of the sender atthe device of the recipient. This is an improvement over existingtechnology that only seeks to protect against unauthorized exposure ofcontent on behalf of the device owner but not on behalf of the sender.

The biometric security mechanism is also advantageous in preventingimpersonation attempts, such as for the purpose of cyber bullying, byverifying the identity of a message sender when composing a message.This improves the overall accountability of messaging within themessaging system. Similarly, in an enterprise environment, theauthentication in both viewing and composition of the message enhancesenterprise-level security and compliance (e.g., eHealth compliance whenthe message content involves medical information).

The disclosure above assumes that the messaging application implementsboth the biometric security mechanism and a messaging interface (e.g.,to compose and read messages). However, it is within the contemplationof this disclosure to implement the biometric security mechanism on aseparate device or application from where the messaging interface isimplemented. For example, in the case of an enterprise email system, anemail access interface may be provided on a desktop computer while thebiometric security mechanism may be implemented on a mobile device(e.g., a smart phone). In this example, the biometric security mechanismcan require the mobile device to connect with the email access interfaceon the desktop computer. The biometric security mechanism can performthe biometric recognition process to verify that there is a secureenvironment near the desktop computer (e.g., no unauthorized user isaround and/or an authorized user is present). In response to verifying asecure environment, the biometric security mechanism can notify theemail access interface to reveal content of a message.

FIGS. 1A through 1C illustrate an example of the biometric securitymechanism that implements a facial recognition process to protect theprivacy of both a sender of a message and a recipient of the message.FIG. 1A is a diagram illustrating a messaging application of a mobiledevice revealing content of a message when a facial profile associatedwith a recipient account is recognized, in accordance with variousembodiments. FIG. 1B is a diagram illustrating a messaging applicationof a mobile device hiding content of a message when a facial profileassociated with a recipient account is not detected, in accordance withvarious embodiments. FIG. 1C is a diagram illustrating a messagingapplication of a mobile device hiding content of a message when a facialprofile associated with a recipient account is detected and a secondunauthorized facial profile is detected, in accordance with variousembodiments.

FIG. 2 is a block diagram of a system environment of a messaging system200 implementing a biometric security mechanism, in accordance withvarious embodiments.

The messaging system 200 can communicate with client devices 202 (e.g.,mobile phones, tablets, desktop computers, laptops, othernetwork-enabled devices, or any combination thereof). The messagingsystem 200 can include a messaging platform system 204 (e.g., one ormore computer servers) configured to provide a service to facilitatehuman-understandable electronic communication between user accounts. Thehuman-understandable electronic communication can include emoticons,text, photos, audio clips, videos, links, images, or any combinationthereof. The human-understandable content of the electroniccommunication may be part of an electronic message or can be referencedin the electronic message (e.g., stored elsewhere that is accessiblethrough a network).

In some embodiments, each of the client devices 202 can have its owninstance of a messaging interface 206 and a corresponding instance of abiometric security engine 207 running thereon that communicates with themessaging platform system 204. In some embodiments, the messaginginterface 206 and the biometric security engine 207 are part of amessaging application running and/or installed on the client device. Insome embodiments, the messaging interface 206 is installed and/orrunning on a first client device and the biometric security engine 207is installed and/or running on a second client device. In theseembodiments, the biometric security engine 207 on the second clientdevice can control a privacy shield implemented by the messaginginterface 206.

For example, the messaging interface 206 and/or the biometric securityengine 207 can be embodied as a mobile application running on operatingsystems of some of the client devices 202. In another example, themessaging interface 206 and/or the biometric security engine 207 can beimplemented as a web-based application running on web browsers on someof the client devices 202.

The client devices 202 can be associated with user accounts. In someembodiments, a user account of the messaging system 200 can havemultiple client devices associated therewith. In some embodiments, aclient device can have multiple user accounts associated therewith.Conversations between user accounts are tracked by the messaging system200 such that the messaging system 200 can deliver an electronic messagefrom a client device of one user account to a client device of anotheruser account.

In some embodiments, the messaging system 200 can include a user profiledatabase 208. The user profile database 208 is configured to store userprofiles of one or more user accounts. The user profiles may beassociated with one or more social networking systems (e.g., anaffiliated social networking system, a social networking systemintegrated with the messaging system 200, or an external socialnetworking system) and social network profiles in the social networkingsystems.

In various embodiments, the messaging interface 206 can implement aprivacy shield. The biometric security engine 207, for example, canrecognize whether or not an authorized user is present by analyzing avideo feed from its respective client device using a facial recognitionalgorithm. The messaging platform system 204 can maintain biometricprofiles of user accounts in the user profile database 208. Themessaging platform system 204 associates a sender account and a receiveraccount with every message (e.g., as specified by the sender account).The biometric profiles of the sender account and the receiver accountcan both be considered “authorized users.” One or more lists ofauthorized users may be stored in an authorization database 210. Theauthorization database 210 can also maintain one or more lists ofblacklisted user accounts that are explicitly unauthorized. In someembodiments, the sender account, the receiver account, or both can addadditional user accounts to the list of authorized users. In someembodiments, the sender account, the receiver account, or both can addone or more user accounts to the list of “blacklisted” user accounts.

The authorization database 210 can store a list of authorized usersspecific to a message conversation or specific to a user account (e.g.,specific to a sender account, a receiver account, or both). For example,for each message or conversation sent or each message or conversationreceived, a user can add or remove one or more accounts from the list ofauthorized users or blacklisted users. For another example, a first useraccount can add a second user account as an authorized user (e.g., aspouse) or a blacklisted user (e.g., a rival friend or sibling) for allconversations that the first user account participates in. In someembodiments (e.g., as required by law) the authorization database 210can also store a list of globally authorized users.

The messaging platform system 204 can communicate with the biometricsecurity engine 207 to secure content of messages. For example, themessaging platform system 204 can send biometric profiles (e.g., facialprofiles) of the authorized users and/or expressly unauthorized users tobe verified on a client device by the biometric security engine 207. Thebiometric security engine 207 can monitor outputs of a sensor 214 (e.g.,a camera) to detect and recognize biometric profiles. In someembodiments, the messaging platform system 204 can request the biometricsecurity engine 207 to capture and send an image or a video of itsoperator to be verified on the messaging platform system 204. In thoseembodiments, the messaging platform system 204 returns the result of theverification back to the biometric security engine 207.

Once the biometric security engine 207 determines that a secureenvironment is present, the biometric security engine 207 can controlthe privacy shield implemented in the messaging interface 206 to eitherreveal or hide content of a message. The privacy shield implemented bythe messaging interface 206 can reveal content of a message conversationwhen the authorized user is detected. In some embodiments, the privacyshield can hide the content in response to detecting both an authorizeduser and an unauthorized user (e.g., implicitly unauthorized orexpressly unauthorized). In some embodiments, the privacy shield canhide the content whenever an expressly unauthorized user account isdetected.

In some embodiments, the client devices 202 can each also include anexpression recognition engine 216. The expression recognition engine 216can configure a client device 202 to execute the disclosed expressionrecognition process. For example, the expression recognition engine 216can implement the methods described in FIG. 3, FIG. 4, and FIG. 5. Insome embodiments, the expression recognition engine 216 can beintegrated with the biometric security engine 207. In some embodiments,the expression recognition engine 216 can be part of the same messagingapplication as the messaging interface 206.

The expression recognition engine 216 can utilize the one or moresensors 214 to monitor at least a biometric signature of an operatinguser. For example, the biometric signature can be a facial profile, ahead profile, a mouth profile, a hand gesture profile, a gait profile, avocal profile, or any combination thereof. From the biometric signature,the expression recognition engine 216 can determine an expression state.The expression state can correspond to an emotion, a mood, or otherpatterns recognizable across the biometric signatures of various people.

In some embodiments, the expression recognition engine 216 can uploadrecognized expression states of an operating user to the messagingplatform system 204. The expression recognition engine 216 can alsoupload associated metadata (e.g., time stamp, associated useridentifier, etc.) to the messaging platform system 204. Based on theexpression states and the associated metadata, the messaging platformsystem 204 can maintain a stimulus response database 218. The stimulusresponse database 218 logs and tracks the expression state of anoperating user in response to an activity occurring on the messaginginterface 206. The activity, for example, can be the operating usercomposing a message, viewing a message, viewing an advertisement,viewing a media object, or any combination thereof. The stimulusresponse database 218 can generate a query interface such that anauthorized agent of the messaging system 200 can access statistic ofspecific users or groups of users.

FIG. 3 is a flow chart of a method 300 of operating a messagingapplication (e.g., the expression recognition engine 216 of FIG. 2) on acomputing device (e.g., one of the client devices 202 of FIG. 2) thatimplements an expression recognition process, in accordance with variousembodiments.

At step 302, the computing device can monitor a video feed from a cameraof the computing device to detect a biometric signature when a messaginginterface of the messaging application is actively being used. At step304, the computing device can match the detected biometric signatureagainst a known profile utilizing a facial recognition process toauthenticate an operating user to use the messaging application. Theknown profile can be stored in the computing device or a messagingserver system (e.g., the messaging platform system 204 of FIG. 2). Insome embodiments, the biometric signature is a facial profile and thebiometric recognition process is a facial recognition process.

As step 306, the computing system can determine a human expression basedon the detected biometric signature utilizing an expression recognitionprocess to associate a contextual tag with an activity on the messaginginterface. At step 308, the computing system can communicate with themessage server system to associate the contextual tag with aconversation involving or participated by the operating user (e.g.,where the operating user engaged in the conversation through themessaging application).

In some cases, the messaging interface is being used to compose amessage in the conversation. In those cases, communicating with themessage server system can include sending the contextual tag as anemoticon embedded as part of the message to configure the message serversystem to distribute the emoticon to another participant of theconversation. In another example, communicating with the message serversystem can include sending the contextual tag as an emotion status thatis time stamped to configure the message server system to maintain anemotion status record of at least a participant of the conversation. Insome cases, the messaging interface is being used to read a message inthe conversation. In those cases, communicating with the message serversystem can include sending the contextual tag as an emotion status thatis time stamped to configure the message server system to present theemotion status to another participant of the conversation. The otherparticipant, for example, can be the writer of the message.

In some cases, communicating with the message server system can includepersonalizing the messaging application based on the contextual tag. Insome cases, communicating with the message server system can includearchiving an expression log associated with the operating user on themessage server system.

Optionally, at step 310, the computing device can activate or deactivatean interactive component of the messaging interface in response todetecting a specific expression via the expression recognition process.That is, biometric expressions (e.g., facial expressions) can be used asa gesture control of the messaging interface/messaging application.

FIG. 4 is a flow chart of a method 400 of operating a messagingapplication (e.g., the expression recognition engine 216 of FIG. 2) on acomputing device (e.g., one of the client devices 202 of FIG. 2) capableof delivering advertisements, in accordance with various embodiments. Atstep 402, the computing device monitors a video feed from a camera(e.g., one of the sensors 214 of FIG. 2) to detect a biometric signaturewhen a messaging interface of the messaging application is activelybeing used. At step 404, the computing device can match the detectedbiometric signature against a known profile utilizing a facialrecognition process to authenticate an operating user to use themessaging application.

At step 406, the computing device determines a human expression based onthe detected biometric signature utilizing an expression recognitionprocess to associate a contextual tag with an activity on the messaginginterface. Optionally, at step 408, the computing device can receive anexpression trigger condition associated with the targeted advertisementfrom a message server system or an advertisement service server. At step410, the computing device presents a targeted advertisement on themessaging interface based on the contextual tag and an identity of theoperating user. The targeted advertisement can be presented in responseto determining that the contextual tag corresponds to the expressiontrigger condition. The expression trigger condition, for example, caninclude a smile, a laugh, a grimace, a frown, a pout, or any combinationthereof. The expression trigger condition can include an expressioncorresponding to a mood or an emotion, a micro expression, a stimulatedexpression, a neutralized expression, a masked expression, or anycombination thereof.

Presenting the targeted advertisement can include selecting the targetedadvertisement from multiple options based on the contextual tag.Selecting the targeted advertisement can be based on a time stamp of thecontextual tag in relation to an activity (e.g., user action ormessaging-app-side activity) occurring on the messaging interface nearor at the time stamp. Selecting the targeted advertisement can be basedon an expression trigger condition associated with the targetedadvertisement.

FIG. 5 is a flow chart of a method 500 of operating a message serversystem (e.g., the messaging platform system 204 of FIG. 2) thatfacilitates conversations between computing devices (e.g., the clientdevices 202 of FIG. 2), in accordance with various embodiments. At step502, the message server system implements a messaging service tofacilitate a messaging application executing on a first computingdevice. At step 504, the message server system authenticates anoperating user of the first computing device based on a facialrecognition process in conjunction with the messaging application whenthe operating user is actively using a messaging interface of themessaging application.

At step 506, the message server system receives an expression statusfrom the messaging application based on an expression recognitionprocess. Receiving the expression status can include receiving metadataassociated with the expression status. The metadata, for example, caninclude a time stamp, a user identifier, a language identifier (e.g.,identifies the language used in the conversation), an ethnic identifierof the user, an associated user action identifier (e.g., what the userwas doing on the messaging interface), user response time, or anycombination thereof.

At step 508, the message server system associates the expression statusin a stimulus response database maintained by the message server system.At step 510, the message server system generates a query interface tothe stimulus response database to provide associated expressions inresponse to a stimulus identifier.

The stimulus identifier, for example, can correspond to a corporateword, a political word, or a social word. The stimulus identifier, foranother example, can correspond to a media object (e.g., image, textstring, audio, video clip, etc.). The stimulus identifier can alsocorrespond to an advertisement.

While processes or methods are presented in a given order, alternativeembodiments may perform routines having steps, or employ systems havingblocks, in a different order, and some processes or blocks may bedeleted, moved, added, subdivided, combined, and/or modified to providealternative or sub-combinations. Each of these processes or blocks maybe implemented in a variety of different ways. In addition, whileprocesses or blocks are at times shown as being performed in series,these processes or blocks may instead be performed in parallel, or maybe performed at different times.

FIG. 6 is a block diagram of an example of a computing device 600, whichmay represent one or more computing devices or servers described herein,in accordance with various embodiments. The computing device 600 can beone or more computing devices that implement the messaging system 200 ofFIG. 2 or methods and processes described in this disclosure. Thecomputing device 600 includes one or more processors 610 and memory 620coupled to an interconnect 630. The interconnect 630 shown in FIG. 6 isan abstraction that represents any one or more separate physical buses,point-to-point connections, or both, connected by appropriate bridges,adapters, or controllers. The interconnect 630, therefore, may include,for example, a system bus, a Peripheral Component Interconnect (PCI) busor PCI-Express bus, a HyperTransport or industry standard architecture(ISA) bus, a small computer system interface (SCSI) bus, a universalserial bus (USB), IIC (I2C) bus, or an Institute of Electrical andElectronics Engineers (IEEE) standard 1494 bus, also called “Firewire”.

The processor(s) 610 is/are the central processing unit (CPU) of thecomputing device 600 and thus controls the overall operation of thecomputing device 600. In certain embodiments, the processor(s) 610accomplishes this by executing software or firmware stored in memory620. The processor(s) 610 may be, or may include, one or moreprogrammable general-purpose or special-purpose microprocessors, digitalsignal processors (DSPs), programmable controllers, application-specificintegrated circuits (ASICs), programmable logic devices (PLDs), trustedplatform modules (TPMs), or the like, or a combination of such devices.

The memory 620 is or includes the main memory of the computing device600. The memory 620 represents any form of random access memory (RAM),read-only memory (ROM), flash memory, or the like, or a combination ofsuch devices. In use, the memory 620 may contain a code 670 containinginstructions according to the mesh connection system disclosed herein.

Also connected to the processor(s) 610 through the interconnect 630 area network adapter 640 and a storage adapter 650. The network adapter 640provides the computing device 600 with the ability to communicate withremote devices over a network and may be, for example, an Ethernetadapter or Fibre Channel adapter. The network adapter 640 may alsoprovide the computing device 600 with the ability to communicate withother computers. The storage adapter 650 enables the computing device600 to access a persistent storage, and may be, for example, a FibreChannel adapter or SCSI adapter.

The code 670 stored in memory 620 may be implemented as software and/orfirmware to program the processor(s) 610 to carry out actions describedabove. In certain embodiments, such software or firmware may beinitially provided to the computing device 600 by downloading it from aremote system through the computing device 600 (e.g., via networkadapter 640).

The techniques introduced herein can be implemented by, for example,programmable circuitry (e.g., one or more microprocessors) programmedwith software and/or firmware, or entirely in special-purpose hardwiredcircuitry, or in a combination of such forms. Special-purpose hardwiredcircuitry may be in the form of, for example, one or moreapplication-specific integrated circuits (ASICs), programmable logicdevices (PLDs), field-programmable gate arrays (FPGAs), etc.

Software or firmware for use in implementing the techniques introducedhere may be stored on a machine-readable storage medium and may beexecuted by one or more general-purpose or special-purpose programmablemicroprocessors. A “machine-readable storage medium,” as the term isused herein, includes any mechanism that can store information in a formaccessible by a machine (a machine may be, for example, a computer,network device, cellular phone, personal digital assistant (PDA),manufacturing tool, any device with one or more processors, etc.). Forexample, a machine-accessible storage medium includesrecordable/non-recordable media (e.g., read-only memory (ROM); randomaccess memory (RAM); magnetic disk storage media; optical storage media;flash memory devices; etc.), etc.

The term “logic,” as used herein, can include, for example, programmablecircuitry programmed with specific software and/or firmware,special-purpose hardwired circuitry, or a combination thereof.

Some embodiments of the disclosure have other aspects, elements,features, and steps in addition to or in place of what is describedabove. These potential additions and replacements are describedthroughout the rest of the specification.

What is claimed is:
 1. A computer-implemented method of operating amessaging application on a computing device, comprising: monitoring acamera feed (e.g., continuous video stream or one or more discretephotographs) from a camera of the computing device to detect a biometricsignature when a messaging interface of the messaging application isactively being used; matching the detected biometric signature against aknown profile utilizing a facial recognition process to authenticate anoperating user to use the messaging application; determining a humanexpression based on the detected biometric signature utilizing anexpression recognition process to associate a contextual tag with anactivity on the messaging interface; and communicating with a messageserver system to associate the contextual tag with content presented tothe operating user or with a conversation participated by the operatinguser via the messaging application.